1. Field of the Invention
The present invention relates, in general, to a separate learning system and method using a two-layered neural network having target values for hidden nodes and, more particularly, to a separate learning system and method using a two-layered neural network having target values for hidden nodes, which set the target values for hidden nodes during separate learning, so that a computational process is separated into an upper connection and a lower connection without changing a network structure and a weight updating rule, thus reducing computational work.
2. Description of the Related Art
Generally, a neural network system has various uses and application fields. For example, a neural network system can be applied and utilized in various fields such as customer management and electronic commerce in data mining, network management, speech recognition, and financial services.
In detail, in data mining fields, Amazon.com and NCOF use a neural network system to manage of customers who purchase books, and to support searches for products on electronic commerce sites. In financial service fields, a neural network system is used to analyze the shape of charts, and to predict tendencies of the price index of stocks. Visa international and Mellon bank in the United States use a neural network system in a general system for detecting the risk of transactions and in a method of picking out persons who are a high credit risk. Further, in the modeling and scientific theory development fields, a neural network system is used to determine conditions such as optimal temperature, pressure, or chemical materials, in a process of manufacturing fluorescent lamps, and is also utilized to detect inverse functions occurring during a manufacturing process in MIT and a simulation process in productivity laboratories.
Learning in a neural network is a process of setting weights to obtain a desired value at an output node that outputs results corresponding to some input. A representative learning method used in a neural network is a backpropagation learning method.
That is, a backpropagation learning method, which is a learning method used in multi-layer and feedforward neural networks, denotes a supervised learning technique. In order to perform learning, input data and desired output data are required.
However, a backpropagation algorithm has convergence problems, such as local minima or plateaus. The plateaus result in the problem of very slow convergence, and the local minima result in a problem in which gradients in all directions equal zero, thus causing the learning process unexpectedly to stop.
Therefore, an arbitrary set of initial weights is problematic in that it cannot guarantee the convergence of network training. In order to solve the above problems, there are methods such as 1) dynamic change of learning rate and momentum, and 2) the selection of a better function for activation or error evaluation based on a new weight updating rule.
Meanwhile, Quick-propagation (QP) and resilient propagation (RPROP) can provide a fast convergence rate, but cannot guarantee convergence to a global minimum.
Further, a genetic algorithm, conjugate gradient and second-order methods, such as Newton's method, require a greater storage space than backpropagation (BP). Therefore, there is a problem in that imbalance exists between convergence stability, required to avoid learning traps in a wide range of parameters, and a convergence speed, or between overall performance and the requirement of a storage space.
In other words, a backpropagation learning method is problematic in that, since it concentrates only on solving the imbalance between convergence speed and convergence stability due to its function, which is to solve the problem in which convergence speed is low and a learning process stalls at a local minimum, thus convergence fails, the backpropagation learning method is not flexible for arbitrary initial weights, cannot guarantee convergence in a wide range of parameters, and cannot solve the problem of local minima and plateaus.